Face recognition based on the multi-scale local image structures

This paper proposes a framework of face recognition based on the multi-scale local structures of the face image. While some basic tools in this framework are inherited from the SIFT algorithm, this work investigates and contributes to all major steps in the feature extraction and image matching. New approaches to keypoint detection, partial descriptor and insignificant keypoint removal are proposed specifically for human face images, a type of non-rigid and smooth visual objects. A strategy of keypoint search for the nearest subject and a two-stage image matching scheme are developed for the face identification task. They circumvent the problem that local structures matched with those in probe disperse into many different gallery images. Although the proposed framework can work for single template per subject, a training procedure is developed for multiple samples per subject. It contains template selection, unstable keypoint removal and template synthesis to meet different requirements in face recognition applications. Each ingredient of the proposed framework is experimentally validated and compared with its counterpart in the SIFT scheme. Results show that the proposed framework outperforms SIFT and some holistic approaches to face recognition. & 2011 Elsevier Ltd. All rights reserved.